An Empirical Comparison of Pruning Methods for Ensemble Classifiers
نویسندگان
چکیده
Many researchers have shown that ensemble methods such as Boosting and Bagging improve the accuracy of classification. Boosting and Bagging perform well with unstable learning algorithms such as neural networks or decision trees. Pruning decision tree classifiers is intended to make trees simpler and more comprehensible and avoid over-fitting. However it is known that pruning individual classifiers of an ensemble does not necessarily lead to improved generalisation. Examples of individual tree pruning methods are Minimum Error Pruning (MEP), Error-based Pruning (EBP), Reduced-Error Pruning(REP), Critical Value Pruning (CVP) and Cost-Complexity Pruning (CCP). In this paper, we report the results of applying Boosting and Bagging with these five pruning methods to eleven datasets.
منابع مشابه
A Preprocessing Technique to Investigate the Stability of Multi-Objective Heuristic Ensemble Classifiers
Background and Objectives: According to the random nature of heuristic algorithms, stability analysis of heuristic ensemble classifiers has particular importance. Methods: The novelty of this paper is using a statistical method consists of Plackett-Burman design, and Taguchi for the first time to specify not only important parameters, but also optimal levels for them. Minitab and Design Expert ...
متن کاملCost Complexity Pruning of Ensemble Classifiers
In this paper we study methods that combine multiple classification models learned over separate data sets in a distributed database setting. Numerous studies posit that such approaches provide the means to efficiently scale learning to large datasets, while also boosting the accuracy of individual classifiers. These gains, however, come at the expense of an increased demand for run-time system...
متن کاملAn empirical solution for over-pruning with a novel ensemble-learning method for fMRI decoding
BACKGROUND Recent functional magnetic resonance imaging (fMRI) decoding techniques allow us to predict the contents of sensory and motor events or participants' mental states from multi-voxel patterns of fMRI signals. Sparse logistic regression (SLR) is a useful pattern classification algorithm that has the advantage of being able to automatically select voxels to avoid over-fitting. However, S...
متن کاملMultilayer Ensemble Pruning via Novel Multi-sub-swarm Particle Swarm Optimization
Recently, classifier ensemble methods are gaining more and more attention in the machine-learning and data-mining communities. In most cases, the performance of an ensemble is better than a single classifier. Many methods for creating diverse classifiers were developed during the past decade. When these diverse classifiers are generated, it is important to select the proper base classifier to j...
متن کاملAn Empirical Investigation on the Use of Diversity for Creation of Classifier Ensembles
We address one of the main open issues about the use of diversity in multiple classifier systems: the effectiveness of the explicit use of diversity measures for creation of classifier ensembles. So far, diversity measures have been mostly used for ensemble pruning, namely, for selecting a subset of classifiers out of an original, larger ensemble. Here we focus on pruning techniques based on fo...
متن کامل